Papers with inference-time method
Let’s Put Ourselves in Sally’s Shoes: Shoes-of-Others Prefilling Improves Theory of Mind in Large Language Models (2026.findings-eacl)
Copied to clipboard
Kazutoshi Shinoda, Nobukatsu Hojo, Kyosuke Nishida, Yoshihiro Yamazaki, Keita Suzuki, Hiroaki Sugiyama, Kuniko Saito
| Challenge: | Existing methods for Theory of Mind (ToM) are specialized for inferring beliefs from contexts involving changes in the world state. |
| Approach: | They propose a method which makes fewer assumptions about contexts and is applicable to broader scenarios. |
| Outcome: | The proposed method makes fewer assumptions about contexts and is applicable to broader scenarios. |
Spotlight Your Instructions: Instruction-following with Dynamic Attention Steering (2026.eacl-long)
Copied to clipboard
| Challenge: | In many real-world applications, users rely on natural language instructions to guide large language models (LLMs) However, LLMs do not attend to these instructions reliably, and users lack simple mechanisms to emphasize their importance beyond modifying prompt wording or structure. |
| Approach: | They propose an inference-time method that enables users to emphasize specific parts of their prompt by steering the model’s attention toward them, aligning the model's perceived importance of different tokens with user intent. |
| Outcome: | The proposed method improves instruction following across tasks involving multiple instructions and generalizes across models of varying scales. |
SH2: Self-Highlighted Hesitation Helps You Decode More Truthfully (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have made great progress in text generation but suffer from hallucinations during reasoning and generation. |
| Approach: | They propose an inference-time method to help LLMs decode truthfully by selecting tokens with the lowest probabilities and concatenating them to the original context. |
| Outcome: | The proposed method improves LLaMA-7b, LLama2-7b and Mistral-7b on hallucination tasks. |
Regularized Contrastive Decoding with Hard Negative Samples for LLM Hallucination Mitigation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models are prone to generate hallucinations, which can undermine their reliability in high-stakes applications. |
| Approach: | They propose a method to capture hallucination signals for mitigating hallucis in large language models by regularizing the model's internal signals to a weaker model . |
| Outcome: | The proposed method achieves better hallucination mitigation performance on four benchmarks. |
RST-Guarder: Enhancing Long-Context Robustness for Safeguards via RST Parsing and Probabilistic Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing guardrail models for harmful-content detection degrade on long-form inputs . Existing models are vulnerable to policy-violating responses, causing false positives based on benign content . |
| Approach: | They propose an inference-time method that improves harmful-content detection for long-form inputs without additional data curation or model training. |
| Outcome: | The proposed method improves harmful-content detection for long-form inputs without additional data curation or model training. |
TELL-TALE: Task Efficient LLMs with Task Aware Layer Elimination (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models typically come with a fixed architecture, but not all layers contribute equally to every downstream task. |
| Approach: | They propose an inference-time method that selectively removes irrelevant or detrimental layers . the method is hardware-agnostic, requires no retraining, and operates entirely at inference time . |
| Outcome: | The proposed method matches or surpasses baseline performance while reducing computational costs. |
RefreshKV: Updating Small KV Cache During Long-form Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating long sequences of tokens are expensive and require memory and computation resources. |
| Approach: | They propose a method that alternates between full context attention and attention over a subset of input tokens during generation. |
| Outcome: | The proposed method achieves comparable speedup to eviction-based methods while improving performance for various long-form generation tasks. |
Activation Steering for Chain-of-Thought Compression (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models produce intermediate explanations, commonly referred to as chains of thought (CoTs), but the generated rationales are typically verbose, consuming many additional tokens, and thus degrading throughput and increasing inference energy consumption. |
| Approach: | They propose to generate concise reasoning traces by directly adjusting internal representations via activation steering. |
| Outcome: | The proposed method reduces generated token length by 69.4% across five reasoning benchmarks while maintaining accuracy. |
Value of Information: A Framework for Human–Agent Communication (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to large language model (LLM) agents fail to account for stakes of different decisions. |
| Approach: | They propose a framework that balances task risk, query ambiguity, user effort . they use a value-of-information framework to dynamically weigh the expected utility gain . |
| Outcome: | The proposed model matches or exceeds the best manually-tuned baselines in four domains . it explicitly balances task risk, query ambiguity, and user effort . |